This invention relates to an approach to distributed processing that is reconfigurable, and more particularly to distributed processing approach that is reconfigurable in response to changes in number of loci of processing or changes in number of data producer or consumer loci.
Distributed processing of a set of data elements may use a number of producers of data elements (i.e., each being a locus of data production), for example, different computer servers, or physical or logical processors within a multiprocessor system, each providing data access (e.g., from a database) or data storage (e.g., from a disk file system) for part of the data to be processed. Similarly, the results of the processing may be sent to a number of loci of data consumption (e.g., processing, storage, transmission), which again may be a set of computer servers or processors. Processing itself may be distributed among different processing loci, for instance each loci being associated with different physical resource such as separate computer servers or processors on servers, or logical resources such as operating system processes on servers, and/or threads of processing within operating system processes. One approach to coordinating the processing is to determine an arrangement of the producer, processing, and consumer loci, for example, in a graph-based data-flow architecture.